INTRODUCTION

Power wheelchairs provide autonomy and promote self-reliance in people with severe mobility impairments. In 1995, powered mobility devices were used by an estimated 291,000 Americans and exhibited growth trends of 5% per year (LaPlante & Kaye, 2010). Based on these projections, there could be well over a million powered mobility devices in use by 2020. Since power wheelchair users may also present accompanying cognitive, sensory, and motor impairments, driving assessments are needed to ensure the safety of both the drivers and their surroundings.

 

Current clinical evaluations are generally qualitative in nature. For example, pediatric wheelchairs are prescribed based on clinical intuition and “trial and error” (Guerette, Tefft, & Furumasu, 2005) or based on assessment batteries, such as the Pediatric Powered Wheelchair Screening Test (Furumasu, Guerette, & Tefft, 2004). For general power wheelchair evaluation, the Wheelchair Skills Test includes a series of common driving tasks that are rated by clinicians (Kirby, Swuste, Dupuis, MacLeod, & Monroe, 2002). To evaluate driving performance more objectively, some studies have begun quantifying joystick metrics with descriptive statistics: reaction time, total driving time, and target accuracy (Dicianno, Spaeth, Cooper, Fitzgerald, & Boninger, 2006); movement error, movement variability, average speed, and average acceleration (Dicianno, Mahajan, Guirand, & Cooper, 2012); and joystick activations, total driving time, and joystick directional variability (Sorrento, Archambault, Routhier, Dessureault, & Boissy, 2011).

However, as a dynamic, stochastic, and nonlinear task (Nechyba & Xu, 1997), human driving contains complexities that are difficult to express with descriptive statistics and ratings. Because driving involves real-time adaptation based on both personal skill and environmental factors, optimal driving decisions are contingent upon each specific individual and the current surroundings. Individual differences (e.g., in fine motor skill) inject nondeterminism into the driving process.

 

To illustrate how this stochastic factor affects optimal control decisions, consider a simplified grid environment in which the goal at any location is to reach the positive block while avoiding the negative block (Figure 1). When a purely deterministic agent makes decisions, those decisions always yield the expected outcome. For example, if it attempts to move forward, it does not have to worry about accidentally moving orthogonally to its left or right. This allows the deterministic agent to use the shortest path distance to its goal. However, when a nondeterministic agent makes decisions, those decisions do not always yield the expected outcome. In this example, the agent has a high probability of moving orthogonally against its intended direction, so it avoids the narrow corridor rather than taking the risk of accidentally falling into the orange block. Although this results in a longer path distance, the nondeterministic agent is still making optimal decisions within the context of its personal skill and situational environment. In stochastic scenarios such as this, the optimal decision always involves a careful balance between the movement probabilities and movement rewards. This type of stochastic decision optimization problem can be formalized as a first-order Markov decision process (MDP).

METHODS

The MDP framework (Russell & Norvig, 2009) consists of a set of states (e.g., wheelchair positions), set of actions (e.g., wheelchair heading vectors), probability distribution of movement transitions (e.g., movement uncertainty), and reward function (e.g., environmental cues) (Figure 2). These components allow us to model the task of driving as a sequence of MDP decisions, accounting for differences in driving skill and environmental contexts. We then develop an individualized performance measure by comparing the driver’s actual actions with those computed from the MDP.

Mapping Wheelchair Headings into MDP Actions

Unlike cars which turn via differential gearing, power wheelchairs turn via differential steering (Figure 3), such that each wheel’s velocity is explicitly governed. For example, the power wheelchair turns to the right by making the left wheel rotate faster than the right wheel. To estimate the driver’s actions, joystick positions are first mapped into wheel velocities (Figure 4) and then converted into heading vectors using rigid body mechanics.

Mapping Uncertainty into MDP Transition Probabilities

Driving is a stochastic process in large part because the result of each decision is always uncertain. Intended decisions do not perfectly manifest themselves in the physical domain, especially for people with impaired fine motor control. The transition probabilities represent an individualized model for a person’s driving tendencies. For this preliminary report, we use a simulated transition model, such that the driver is assumed to have an 80% probability of completing the intended action (Figure 5).

Mapping Environmental Context into MDP Rewards

We use the Kinect sensor (Microsoft Corporation, Redmond, WA) to sense the environment. The Kinect captures color and depth images within a 57° horizontal field of view and 4,000 mm radial distance. Based on the joystick actions and movement transitions, we model the MDP’s states as a 3-branch, 6-depth graph (Figure 6). Each branch represents one of the three joystick actions (left, middle, right) that can be chosen at a given state

Optimizing the MDP Policy

In the MDP framework, decisions can be optimized into a policy of actions that maximizes the agent’s expected utility using a simplified Bellman equation (Equation 1). Optimization of the utility values is performed using backward induction (Russell & Norvig, 2009).

U(s) = R(s) + max_a Σ_s'P(s'|s,a)U(s')     (1)

Equation 1: U(s) is the utility value for a state s, R(s) is the reward for a state s, and P(s'|s,a) is the transition probability of reaching state s' given the current state s and action a.

Assessing the Driving Performance

Although the backward induction algorithm computes a utility-optimal policy, there is nothing inherently wrong with suboptimal decisions. Mathematically, it simply means that the driver can no longer achieve the highest possible utility. Practically, the impact could range anywhere from negligible to catastrophic. The severity of the deviations can be quantified in terms of the resulting expected utility.

At the root node of each graph, the computed and observed actions are compared based on their resulting expected utilities. A risk index is calculated by normalizing the utility difference against the optimal utility (Equation 2). Thus, when the computed and observed actions match, the risk index evaluates to zero; otherwise, riskier mismatches evaluate to higher values.

I(t) = [U_Obs(t) - U_MDP (t)]/U_MDP(t) (2)

Equation 2: I(t) is the risk index at time t, U_Obs(t) is the utility value of the observed action at the root node at time t, and U_MDP(t) is the utility value of the MDP action at the root node at time t.

RESULTS

We performed preliminary experiments on a C400 power wheelchair (Permobil, Inc., Lebanon, TN). Figure 7 presents a selected time series of the calculated risk indices. Sample camera images, which have been superimposed with computed and observed heading vectors, are shown for both high-risk (Figures 8) and low-risk (Figure 9) situations.

DISCUSSION

We used a simulated transition model that assumed the driver to possess a relatively high probability (20%) of mistakenly moving to the right or left of each intended direction. Figure 8 shows an example of a high-risk mismatch between the computed MDP action (move left) and the observed action (move forward). For an able-bodied driver, the observed action might not be considered risky. However, the MDP was provided a simulated model of a relatively unpredictable driver, causing the optimization algorithm to compute a left action. For such a driver, attempting to move straight might accidentally cause the driver to move to the right and collide with the extremely proximate door. Figure 9 illustrates another mismatch. Again, the MDP accounted for the model’s driving uncertainty and computed a conservative optimal policy to perform a right turn away from the wall. However, the risk index was negligible. In this case, the wall was more distant than in Figure 8, so the expected utility difference between the mismatched actions was lower. These examples illustrate the model’s ability to account for both driving tendencies and environmental cues.

Although our reported examples focused on driving decisions, the value of individualized driving evaluation extends beyond the explicit path decisions. For example, a driver may actually be executing optimal driving decisions, even if the joystick activation counts or driving durations are not absolutely minimal. As depicted earlier in the simplified grid example, optimal paths may be altered significantly based on a driver’s tendencies. Perhaps a driver, caregiver, or clinician has noticed that the driver frequently pulls to the left accidentally. Such a driver might instinctively stay farther from the left wall and/or make extremely wide left turns, but these inefficiencies may actually be desirable.

Our MDP quantification method provides awareness of situational context (i.e., reward function via the Kinect data) and individual differences (i.e., transition model). In this study, we used a simulated transition model as a demonstration of the method. In future work, we plan to empirically estimate the transition model of each driver’s joystick control, allowing the MDP to compute individualized optimal policies as a reference to evaluate each driver’s performance.